Papers with fine-tuning large language models
Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Traditionally, success in multilingual machine translation depends on large volume, diverse directions, and high quality of training data. |
| Approach: | They revisit the importance of large language models for translation by fine-tuning on 32 parallel sentences. |
| Outcome: | The proposed model can be fine-tuned on as few as 32 parallel sentences . however, the choice of direction is critical to avoid misinterpretation, the authors say . |
RL with KL penalties is better viewed as Bayesian inference (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) is used in fine-tuning large language models to penalize them for undesirable features of generated sequences. |
| Approach: | They analyze challenges associated with treating a language model as an RL policy . they find that RL is equivalent to variational inference: approximating a Bayesian posterior . |
| Outcome: | The proposed approach is flawed because it turns the LM into a degenerate distribution, the authors show . they show that the proposed approach avoids the distribution collapse problem and offers a first-principles derivation for its objective. |
Analyzing Effects of Learning Downstream Tasks on Moral Bias in Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models replicate and perpetuate social biases . pre-existing moral bias may be mitigated or amplified even when presented with opposing views . |
| Approach: | They develop methods to assess the agreement of LMs to explicit codified norms . they find that introducing downstream tasks may lead to unexpected inconsistencies . |
| Outcome: | The proposed model can be used to improve morality in data-scarce tasks. |
TeRA: Vector-based Random Tensor Network for High-Rank Adaptation of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) methods have significantly reduced the number of trainable parameters needed in fine-tuning large language models. |
| Approach: | They propose a vector-based random Tensor network for high-Rank Adaptation method that achieves high-rank weight updates while retaining parameter efficiency. |
| Outcome: | The proposed method outperforms existing PEFT methods while keeping low-rank parameters. |
LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild (2024.findings-acl)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) is an effective yet efficient solution for fine-tuning large language models. |
| Approach: | They propose a low-rank Adaptation framework that retrieves and composes multiple LoRAs according to input prompts. |
| Outcome: | Experimental results show that LoraRetriever outperforms baselines in terms of performance and versatility. |
Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies focus on data selection but lack a clear, unified framework . variability in experimental settings complicates systematic comparisons . |
| Approach: | They propose a three-stage scheme to standardize data selection for fine-tuning large language models . they propose unified comparison approach that incorporates ratio-based efficiency and ranking-based feasibility metrics to address inconsistencies across experiments. |
| Outcome: | The proposed scheme outperforms existing methods in a dozen key studies and identifies key challenges. |
MUZO: Leveraging Multiple Queries and Momentum for Zeroth-Order Fine-Tuning of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models incur memory overhead due to the need for activation storage for back-propagation (BP). |
| Approach: | They propose a method that estimates gradients through finite differences without activation storage for back-propagation. |
| Outcome: | The proposed method demonstrates superior performance in fine-tuning various LLMs. |
LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Low-rank adaptation (LoRA) fine-tunes large language models due to its significant reduction in trainable parameters, but its backward updates require storing high-dimensional intermediate activations and optimizer states, requiring high peak GPU memory. |
| Approach: | They propose a low-dimensional adaptation approach to fine-tune large language models which freezes a first projection matrix while introducing a lower-dimensional trainable square matrix. |
| Outcome: | The proposed approach reduces trainable parameters and peak GPU memory footprint while preserving low-dimensional trainable square matrix. |
Making Large Language Models Efficient Dense Retrievers (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that fine-tuning large language models for dense retrieval yields strong performance, but their substantial parameter counts make them computationally inefficient. |
| Approach: | They propose a framework for developing efficient retrievers that performs coarse-to-fine compression through a coarse-grained coarse-tuning strategy. |
| Outcome: | The proposed framework reduces model size and inference cost while preserving performance of full-size models. |
PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in fine-tuning large language models have greatly enhanced their usage in domain-specific tasks. |
| Approach: | They propose a method which internalizes prompt knowledge during model fine-tuning to achieve efficient inference and save costs. |
| Outcome: | The proposed approach reduces input tokens by 90%, accelerates inference by 4.2 times, and reduces monetary inference costs by 88.3%. |
Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuning (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work on predicting backchannel timing has focused on lexical form and prosody, but the relationship between lexico-prosodic form and meaning remains underexplored. |
| Approach: | They propose a framework for fine-tuning large language models on dialogue transcripts to derive rich contextual representations; and a joint embedding space for dialogue contexts and backchannel realizations. |
| Outcome: | The proposed framework improves context-backchannel retrieval and human perception is more sensitive to extended conversational context and embeddings align more closely with human judgments than raw WavLM features. |
Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Models (2025.acl-long)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) is a common method for fine- tuning large language models . however, once updated, PEFT modules suffer performance degradation on newer versions . |
| Approach: | They propose a method that enhances the PEFT module by focusing on the task-specific pattern while reducing its dependence on certain knowledge in the base model. |
| Outcome: | Experiments show that PEFT modules can maintain performance on updated models without re-tuning . the proposed approach can be used in real-world applications with large model sizes . |
Power doesn’t reside in size: A Low Parameter Hybrid Language Model (HLM) for Sentiment Analysis in Code-mixed data (2025.emnlp-main)
Copied to clipboard
Pavan Sai Balaga, Nagasamudram Karthik, Challa Vishwanath, Raksha Sharma, Rudra Murthy, Ashish Mittal
| Challenge: | Code-mixed text presents significant challenges for machine learning due to interplay of distinct grammatical structures, effectively forming a hybrid language. |
| Approach: | They propose a Hybrid Language Model that combines a multilingual encoder and a lightweight decoder to achieve sentiment classification performance comparable to those of fine-tuned Large Language Models. |
| Outcome: | The proposed model outperforms models trained individually in sentiment detection tasks. |
GCIG: GraphRAG-based Cross-document Instruction Generation for Boosting LLM Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models struggle in knowledge-intensive domains and complex reasoning tasks due to their limited coverage of single-document knowledge and repetitive content. |
| Approach: | They propose a GraphRAG-based cross-document instruction generation framework that generates diverse questions through task-aware prompts and context-sensitive retrieval. |
| Outcome: | The proposed framework outperforms existing methods on knowledge-intensive and multi-hop question-answering tasks. |
Annotation-Efficient Language Model Alignment via Diverse and Representative Response Texts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | obtaining large amount of preference annotations is difficult in many applications . obtaining a large amount is difficult, so a preference dataset needs limited annotation budget . |
| Approach: | They propose annotating preference over a subset of responses that maximizes diversity and representativeness from available responses and then annotates preference over the selected ones. |
| Outcome: | The proposed method outperforms baselines with the same annotation budget. |
LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging (2026.acl-long)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) is a parameter-efficient approach for fine-tuning large language models. |
| Approach: | They propose a low-rank Adaptation framework that automatically selects and merges LoRA adapters at the instance level without additional training. |
| Outcome: | The proposed framework outperforms training-based baselines on some tasks upto a margin of 3.6% while remaining competitive on other tasks and maintaining inference throughput. |